Cluster-based Input Weight Initialization for Echo State Networks

Echo State Networks (ESNs) are a special type of recurrent neural networks (RNNs), in which the input and recurrent connections are traditionally generated randomly, and only the output weights are trained. Despite the recent success of ESNs in various tasks of audio, image and radar recognition, we postulate that a purely random initialization is not the ideal way of initializing ESNs. The aim of this work is to propose an unsupervised initialization of the input connections using the K-Means algorithm on the training data. We show that this initialization performs equivalently or superior than a randomly initialized ESN whilst needing significantly less reservoir neurons (2000 vs. 4000 for spoken digit recognition, and 300 vs. 8000 neurons for f0 extraction) and thus reducing the amount of training time. Furthermore, we discuss that this approach provides the opportunity to estimate the suitable size of the reservoir based on the prior knowledge about the data.Comment: Submitted to IEEE Transactions on Neural Network and Learning System (TNNLS), 202

PyRCN: A Toolbox for Exploration and Application of Reservoir Computing Networks

Reservoir Computing Networks belong to a group of machine learning techniques that project the input space non-linearly into a high-dimensional feature space, where the underlying task can be solved linearly. Popular variants of RCNs, e.g.\ Extreme Learning Machines (ELMs), Echo State Networks (ESNs) and Liquid State Machines (LSMs) are capable of solving complex tasks equivalently to widely used deep neural networks, but with a substantially simpler training paradigm based on linear regression. In this paper, we introduce the Python toolbox PyRCN (Python Reservoir Computing Networks) for optimizing, training and analyzing Reservoir Computing Networks (RCNs) on arbitrarily large datasets. The tool is based on widely-used scientific packages, such as numpy and scipy and complies with the scikit-learn interface specification. It provides a platform for educational and exploratory analyses of RCNs, as well as a framework to apply RCNs on complex tasks including sequence processing. With only a small number of basic components, the framework allows the implementation of a vast number of different RCN architectures. We provide extensive code examples on how to set up RCNs for a time series prediction and for a sequence classification task.Comment: Preprint submitted to Engineering Applications of Artificial Intelligenc

Control concepts for articulatory speech synthesis

We present two concepts for the generation of gestural scores to control an articulatory speech synthesizer. Gestural scores are the common input to the synthesizer and constitute an or- ganized pattern of articulatory gestures. The first concept gen- erates the gestures for an utterance using the phonetic transcrip- tions, phone durations, and intonation commands predicted by the Bonn Open Synthesis System (BOSS) from an arbitrary in- put text. This concept extends the synthesizerto a text-to-speech synthesis system. The idea of the second concept is to use tim- ing informationextracted from ElectromagneticArticulography signals to generate the articulatory gestures. Therefore, it is a concept for the re-synthesis of natural utterances. Finally, ap- plication prospects for the presented synthesizer are discussed

Dielectrophoresis: An Approach to Increase Sensitivity, Reduce Response Time and to Suppress Nonspecific Binding in Biosensors?

The performance of receptor-based biosensors is often limited by either diffusion of the analyte causing unreasonable long assay times or a lack of specificity limiting the sensitivity due to the noise of nonspecific binding. Alternating current (AC) electrokinetics and its effect on biosensing is an increasing field of research dedicated to address this issue and can improve mass transfer of the analyte by electrothermal effects, electroosmosis, or dielectrophoresis (DEP). Accordingly, several works have shown improved sensitivity and lowered assay times by order of magnitude thanks to the improved mass transfer with these techniques. To realize high sensitivity in real samples with realistic sample matrix avoiding nonspecific binding is critical and the improved mass transfer should ideally be specific to the target analyte. In this paper we cover recent approaches to combine biosensors with DEP, which is the AC kinetic approach with the highest selectivity. We conclude that while associated with many challenges, for several applications the approach could be beneficial, especially if more work is dedicated to minimizing nonspecific bindings, for which DEP offers interesting perspectives

Micro-electromechanical affinity sensor for the monitoring of glucose in bioprocess media

An affinity-viscometry-based micro-sensor probe for continuous glucose monitoring was investigated with respect to its suitability for bioprocesses. The sensor operates with glucose and dextran competing as binding partner for concanavalin A, while the viscosity of the assay scales with glucose concentration. Changes in viscosity are determined with a micro-electromechanical system (MEMS) in the measurement cavity of the sensor probe. The study aimed to elucidate the interactions between the assay and a typical phosphate buffered bacterial cultivation medium. It turned out that contact with the medium resulted in a significant long-lasting drift of the assay’s viscosity at zero glucose concentration. Adding glucose to the medium lowers the drift by a factor of eight. The cglc values measured off-line with the glucose sensor for monitoring of a bacterial cultivation were similar to the measurements with an enzymatic assay with a difference of less than ±0.15 g·L−1. We propose that lectin agglomeration, the electro-viscous effect, and constitutional changes of concanavalin A due to exchanges of the incorporated metal ions may account for the observed viscosity increase. The study has demonstrated the potential of the MEMS sensor to determine sensitive viscosity changes within very small sample volumes, which could be of interest for various biotechnological applications.DFG, 325093850, Open Access Publizieren 2017 - 2018 / Technische Universität Berli

Separation, Characterization, and Handling of Microalgae by Dielectrophoresis

Microalgae biotechnology has a high potential for sustainable bioproduction of diverse high-value biomolecules. Some of the main bottlenecks in cell-based bioproduction, and more specifically in microalgae-based bioproduction, are due to insufficient methods for rapid and efficient cell characterization, which contributes to having only a few industrially established microalgal species in commercial use. Dielectrophoresis-based microfluidic devices have been long established as promising tools for label-free handling, characterization, and separation of broad ranges of cells. The technique is based on differences in dielectric properties and sizes, which results in different degrees of cell movement under an applied inhomogeneous electrical field. The method has also earned interest for separating microalgae based on their intrinsic properties, since their dielectric properties may significantly change during bioproduction, in particular for lipid-producing species. Here, we provide a comprehensive review of dielectrophoresis-based microfluidic devices that are used for handling, characterization, and separation of microalgae. Additionally, we provide a perspective on related areas of research in cell-based bioproduction that can benefit from dielectrophoresis-based microdevices. This work provides key information that will be useful for microalgae researchers to decide whether dielectrophoresis and which method is most suitable for their particular application.BMBF, 031B0381, IBÖ-04: SepaDiElo - Mikroelektronik-System zur Zellseparatio

Improved acoustic modeling for automatic piano music transcription using echo state networks

Automatic music transcription (AMT) is one of the challenging problems in Music Information Retrieval with the goal of generating a score-like representation of a polyphonic audio signal. Typically, the starting point of AMT is an acoustic model that computes note likelihoods from feature vectors. In this work, we evaluate the capabilities of Echo State Networks (ESNs) in acoustic modeling of piano music. Our experiments show that the ESN-based models outperform state-of-the-art Convolutional Neural Networks (CNNs) by an absolute improvement of 0.5 F-1-score without using an extra language model. We also discuss that a two-layer ESN, which mimics a hybrid acoustic and language model, achieves better results than the best reference approach that combines Invertible Neural Networks (INNs) with a biGRU language model by an absolute improvement of 0.91 F-1-score

Separation, characterization, and handling of microalgae by dielectrophoresis

Microalgae biotechnology has a high potential for sustainable bioproduction of diverse highvalue biomolecules. Some of the main bottlenecks in cell-based bioproduction, and more specifically in microalgae-based bioproduction, are due to insufficient methods for rapid and efficient cell characterization, which contributes to having only a few industrially established microalgal species in commercial use. Dielectrophoresis-based microfluidic devices have been long established as promising tools for label-free handling, characterization, and separation of broad ranges of cells. The technique is based on differences in dielectric properties and sizes, which results in different degrees of cell movement under an applied inhomogeneous electrical field. The method has also earned interest for separating microalgae based on their intrinsic properties, since their dielectric properties may significantly change during bioproduction, in particular for lipid-producing species. Here, we provide a comprehensive review of dielectrophoresis-based microfluidic devices that are used for handling, characterization, and separation of microalgae. Additionally, we provide a perspective on related areas of research in cell-based bioproduction that can benefit from dielectrophoresis-based microdevices. This work provides key information that will be useful for microalgae researchers to decide whether dielectrophoresis and which method is most suitable for their particular application. © 2020 by the authors. Licensee MDPI, Basel, Switzerland